SCAN: Spectral Compressed Analysis for Monitoring Evolving Multi-Relational Social Networks

We propose SCAN, an innovative, spectral analysis framework for internet scale monitoring of multi-relational social media data, encoded in the form of tensor streams. In particular, a significant challenge is to detect key changes in the social media data, which could reflect important events in the real world, sufficiently quickly. Social media data have three challenging characteristics. First, data sizes are enormous – recent technological advances allow hundreds of millions of users to create and share content within online social networks. Second, social data are often multi-faceted (i.e., have many dimensions of potential interest, from the textual content to user metadata). Finally, the data is dynamic – changes can occur at multiple time scales and be localized to a subset of users. Consequently, framework for extracting useful information from social media data needs to scale with data volume, and also against the number and diversity of the facets of the data. In SCAN, we focus on the computational cost of structural change detection in tensor streams. We introduce compressed sensing, which through the use of randomized tensor ensembles, is able to encode the observed tensor streams in the form of compact descriptors. We show that the descriptors allow very fast detection of significant spectral changes in the tensor stream, which also reduce data collection, storage, and processing costs. Experiments over synthetic and real tensor stream data, show that SCAN is faster (17.7x–159x for change detection) and more accurate (above 0.9 F-score) than baseline methods.